Holistic Regression Testing for High-Quality MT
نویسندگان
چکیده
We review the techniques and tools used for regression testing, the primary quality assurance measure, in a multi-site research project working towards a high-quality Norwegian –English MT demonstrator. A combination of hand-constructed test suites, domain-specific corpora, specialized software tools, and somewhat rigid release procedures is used for semi-automated diagnostic and regression evaluation. Based on project-internal experience so far, we comment on a range of methodological aspects and desiderata for systematic evaluation in MT development and show analogies to evaluation work in other NLP tasks.
منابع مشابه
Holistic use of analysis models in model-based system testing
Nowadays software testing techniques have to fulfill the requirements of growing complexity of evolving software systems. To handle the requirements current research is strongly interested in the field of model-based testing (MBT). While MBT is becoming the next generation of testing, using it in practice at the system test level two main problems arise: missing use of analysis models for testi...
متن کاملMachine Translation Quality Estimation Adapted to the Translation Workflow
The varying quality of machine translation (MT) poses a problem for language service providers (LSPs) which want to use MT to make the translation production process more efficient. In this user study we describe the MT confidence score we developed. It predicts the quality of a segment translated by MT and it is fully integrated into the translation workflow.
متن کاملGray Box Coverage Criteria for Testing Graph Pattern Matching
Model transformations (MT) are a core building block of Model-Driven Engineering. The quality of MT specifications and implementations is vital to their success. The well-researched formal underpinning of graph transformation (GT) theory allows for proving quality-relevant properties and enables stringent implementations. Yet, in practice, MT implementations often depend on verification/validat...
متن کاملMachine learning methods for comparative and time-oriented Quality Estimation of Machine Translation output
This paper describes a set of experiments on two sub-tasks of Quality Estimation of Machine Translation (MT) output. Sentence-level ranking of alternative MT outputs is done with pairwise classifiers using Logistic Regression with blackbox features originating from PCFG Parsing, language models and various counts. Post-editing time prediction uses regression models, additionally fed with new el...
متن کاملSelecting Feature Sets for Comparative and Time-Oriented Quality Estimation of Machine Translation Output
This paper describes a set of experiments on two sub-tasks of Quality Estimation of Machine Translation (MT) output. Sentence-level ranking of alternative MT outputs is done with pairwise classifiers using Logistic Regression with blackbox features originating from PCFG Parsing, language models and various counts. Post-editing time prediction uses regression models, additionally fed with new el...
متن کامل